Papers with navigation tasks
NavA3: Understanding Any Instruction, Navigating Anywhere, Finding Anything (2026.acl-long)
Copied to clipboard
Lingfeng Zhang, Xiaoshuai Hao, Yingbo Tang, Haoxiang Fu, Xinyu Zheng, Pengwei Wang, Zhongyuan Wang, Wenbo Ding, Shanghang Zhang
| Challenge: | Existing embodied navigation methods struggle with such tasks due to their limitations in comprehending high-level human instructions and localizing objects with an open vocabulary. |
| Approach: | They propose a hierarchical framework for long-horizon navigation that integrates human instructions with 3D scene views. |
| Outcome: | The proposed model achieves SOTA results and can complete long-horizon navigation tasks across different robot embodiments in real-world environments. |
When Can Transformers Ground and Compose: Insights from Compositional Generalization Benchmarks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent benchmarks like ReaSCAN use navigation tasks grounded in a grid world to assess whether neural models exhibit compositional behaviour. |
| Approach: | They propose a transformer-based model that outperforms specialized architectures on ReaSCAN and a modified version of gSCAN to test their performance. |
| Outcome: | The proposed model outperforms specialized architectures on ReaSCAN and gSCAN on a grid world and can generalize to deeper input structures. |
Dialogue-AMR: Abstract Meaning Representation for Dialogue (2020.lrec-1)
Copied to clipboard
Claire Bonial, Lucia Donatelli, Mitchell Abrams, Stephanie M. Lukin, Stephen Tratz, Matthew Marge, Ron Artstein, David Traum, Clare Voss
| Challenge: | Abstract Meaning Representation (AMR) does not capture the illocutionary force or speaker’s intended contribution in the broader dialogue context. |
| Approach: | They propose a schema that enriches Abstract Meaning Representation (AMR) it provides a semantic representation for facilitating Natural Language Understanding (NLU) in dialogue systems. |
| Outcome: | The proposed schema provides a semantic representation for facilitating Natural Language Understanding (NLU) in human-robot dialogue systems. |
BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps (2020.acl-main)
Copied to clipboard
| Challenge: | Existing state-of-the-art VLN agents do not generalize well for long navigation tasks. |
| Approach: | They propose a VLN agent that is learned to navigate by decomposing long instructions into shorter ones and completing them sequentially. |
| Outcome: | The proposed agent can follow long instructions better than existing ones, but it does not generalize well. |
Masked Path Modeling for Vision-and-Language Navigation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | A major challenge in vision-and-language navigation is the limited available training data, which hinders the models’ ability to generalize effectively. |
| Approach: | They propose a masked path modeling objective that pretrains an agent using self-collected data for subsequent navigation tasks. |
| Outcome: | The proposed model pretrains an agent using self-collected data for subsequent navigation tasks eliminating the need for external tools. |
SCOUT: A Situated and Multi-Modal Human-Robot Dialogue Corpus (2024.lrec-main)
Copied to clipboard
Stephanie M. Lukin, Claire Bonial, Matthew Marge, Taylor A. Hudson, Cory J. Hayes, Kimberly Pollard, Anthony Baker, Ashley N. Foots, Ron Artstein, Felix Gervits, Mitchell Abrams, Cassidy Henry, Lucia Donatelli, Anton Leuski, Susan G. Hill, David Traum, Clare Voss
| Challenge: | The corpus contains 89,056 utterances and 310,095 words from 278 dialogues averaging 320 utterrances per dialogue. |
| Approach: | They present the Situated Corpus Of Understanding Transactions, a multi-modal collection of human-robot dialogue in the task domain of collaborative exploration. |
| Outcome: | The Situated Corpus Of Understanding Transactions (SCOUT) contains 89,056 utterances and 310,095 words from 278 dialogues averaging 320 utterrances per dialogue. |
VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models have demonstrated remarkable capabilities across vision-language tasks, but their performance as embodied agents needs further exploration. |
| Approach: | They propose a framework to evaluate multimodal large language models as zero-shot agents . they find that enhancing prevalent agents with Chain-of-Thought reasoning and self-reflection leads to an unexpected performance decrease. |
| Outcome: | The proposed framework enables comparisons and component-level ablations across diverse MLLM architectures, agent designs, and navigation tasks. |
GraphMind: LLMs as Dynamic Knowledge Builders for Sequential Decision-Making (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance in natural language understanding and generation, establishing themselves as foundational tools across a wide range of domains. |
| Approach: | They propose an LLM agent architecture that integrates a knowledge graph as a graph-based memory module and integrates it into the agent to generate efficient plans. |
| Outcome: | The proposed architecture improves the performance and efficiency of the LLM in navigation tasks designed to present long-horizon and partially observable challenges. |